Rational Coordination in Multi-Agent Environments

We adopt the decision-theoretic principle of expected utility maximization as a paradigm for designing autonomous rational agents, and present a framework that uses this paradigm to determine the choice of coordinated action. We endow an agent with a specialized representation that captures the agent's knowledge about the environment and about the other agents, including its knowledge about their states of knowledge, which can include what they know about the other agents, and so on. This reciprocity leads to a recursive nesting of models. Our framework puts forth a representation for the recursive models and, under the assumption that the nesting of models is finite, uses dynamic programming to solve this representation for the agent's rational choice of action. Using a decision-theoretic approach, our work addresses concerns of agent decision-making about coordinated action in unpredictable situations, without imposing upon agents pre-designed prescriptions, or protocols, about standard rules of interaction. We implemented our method in a number of domains and we show results of coordination among our automated agents, among human-controlled agents, and among our agents coordinating with human-controlled agents.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/44002/1/10458_2004_Article_272540.pd

Rational Communication in Multi-Agent Environments

We address the issue of rational communicative behavior among autonomous self-interested agents that have to make decisions as to what to communicate, to whom, and how. Following decision theory, we postulate that a rational speaker should design a speech act so as to optimize the benefit it obtains as the result of the interaction. We quantify the gain in the quality of interaction in terms of the expected utility, and we present a framework that allows an agent to compute the expected utilities of various communicative actions. Our framework uses the Recursive Modeling Method as the specialized representation used for decision-making in a multi-agent environment. This representation includes information about the agent's state of knowledge, including the agent's preferences, abilities and beliefs about the world, as well as the beliefs the agent has about the other agents, the beliefs it has about the other agents' beliefs, and so on. Decision-theoretic pragmatics of a communicative act can be then defined as the transformation the act induces on the agent's state of knowledge about its decision-making situation. This transformation leads to a change in the quality of interaction, expressed in terms of the expected utilities of the agent's best actions before and after the communicative act. We analyze decision-theoretic pragmatics of a number of important kinds of communicative acts and investigate their expected utilities using examples. Finally, we report on the agreement between our method of message selection and messages that human subjects choose in various circumstances, and show an implementation and experimental validation of our framework in a simulated multi-agent environment.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/44016/1/10458_2004_Article_350961.pd

Toward a theory of honesty and trust among communicating autonomous agents

This article outlines, through a number of examples, a method that can be used by autonomous agents to decide among potential messages to send to other agents, without having to assume that a message must be truthful and that it must be believed by the hearer. The main idea is that communicative behavior of autonomous agents is guided by the principle of economic rationality, whereby agents transmit messages to increase the effectiveness of interaction measured by their expected utilities. We are using a recursive, decision-theoretic formalism that allows agents to model each other and to infer the impact of a message on its recipient. The recursion can be continued into deeper levels, and agents can model the recipient modeling the sender in an effort to assess the truthfulness of the received message. We show how our method often allows the agents to decide to communicate in spite of the possibility that the messages will not be believed. In certain situations, on the other hand, our method shows that the possibility of the hearer not believing what it hears makes communication useless. Our method thus provides the rudiments of a theory of how honesty and trust could emerge through rational, selfish behavior.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/42827/1/10726_2005_Article_BF01384248.pd

A framework for sequential planning in multi-agent settings

This paper extends the framework of partially observable Markov decision processes (POMDPs) to multi-agent settings by incorporating the notion of agent models into the state space. Agents maintain beliefs over physical states of the environment and over models of other agents, and they use Bayesian update to maintain their beliefs over time. The solutions map belief states to actions. Models of other agents may include their belief states and are related to agent types considered in games of incomplete information. We express the agents ’ autonomy by postulating that their models are not directly manipulable or observable by other agents. We show that important properties of POMDPs, such as convergence of value iteration, the rate of convergence, and piece-wise linearity and convexity of the value functions carry over to our framework. Our approach complements a more traditional approach to interactive settings which uses Nash equilibria as a solution paradigm. We seek to avoid some of the drawbacks of equilibria which may be non-unique and are not able to capture off-equilibrium behaviors. We do so at the cost of having to represent, process and continually revise models of other agents. Since the agent’s beliefs may be arbitrarily nested the optimal solutions to decision making problems are only asymptotically computable. However, approximate belief updates and approximately optimal plans are computable. We illustrate our framework using a simple application domain, and we show examples of belief updates and value functions. 1

An Approach to User Modeling in Decision Support Systems

. Drawing on our work in the area of distributed artificial intelligence, we put forth a framework for modeling a human user interacting with a knowledge-based system. We assume that the human user is situated in some decision making setting, and view the computer system as taking an active role in supporting the user's decision making and problem solving activities. The model the system has of the decision making situation and of the user can be applied to determine what the system should do, both in terms of the system's physical action, if such is possible, as well as in terms of the information that should be transmitted to the user. An important part of the user's model is the model that the user may have of the system itself, and, further, how the user may think it is being modeled by the system. Our framework, the Recursive Modeling Method (RMM), explicitly represents this nesting of models, and lets the system to coordinate with the expected actions of the human user, and to r..

Rational communicative behavior in anti-air defense

1 1 Introduction This paper describes the decision-theoretic message selection of an automated agent engaged in an anti-air defense with other defense agents. The goal of defending agents is to minimize damages to their territory [10]. To fulfill their mission, the agents need to coordinate and, sometimes, to communicate with other agents. However, since the communication bandwidth is usually limited in a battlefield environment, and disclosure of any information to hostile agents should be avoided, it is critical for a defending agent to be selective as to what messages should be sent to other agents